241 architecture extend quality control configuration - #242
Merged
roryclaydon1994 merged 12 commits intoAug 18, 2026
Merged
Conversation
10 tasks
added 4 commits
August 12, 2026 12:36
- Add resources section - Add required_resources option for metrics - Add params section with example parameters
- Add dataclass ResourcesConfig to store resources available parameters - Add method parse_resources_config to parse resources config section - Add validators to check for internal consistency between resources, metrics and rejection params - Normalise doc string formats - Add missing doc strings for parser methods - Update changelog fragment re: quality control configuration framework
- Update/add config fixtures with resource sections - Update existing tests to evaluate resource sections - Add unit tests for parse resource section and validators - Add integration test for loader that runs validator
jeipollack
force-pushed
the
241-architecture-extend-quality-control-configuration
branch
from
August 12, 2026 10:37
a5c1df3 to
b0ab44f
Compare
roryclaydon1994
left a comment
roryclaydon1994
left a comment
There was a problem hiding this comment.
Review Feedback
Great job, architecture clean and introduces minimal coupling and tests are comprehensive. Please see inline comments.
added 5 commits
August 17, 2026 14:45
- Accept Mapping inputs instead of concrete dictionaries. - Normalize mapped configuration data to dictionaries in the parsed configuration. - Update parser terminology and docstrings accordingly.
- Simplify resource identifier validation into a single conditional. - Reject identifiers containing more than one "." separator. - Consolidate identifier validation tests using pytest.mark.parametrize. - Consolidate unknown resource validation tests using pytest.mark.parametrize.
- Update parse_metrics_config to require required_resources to be a list. - Add a TypeError test and fixture for a non-list required_resources value. - Add a TypeError test and fixture for a non-string required_resources element. - Remove the deprecated fixture YAML file.
Contributor
Author
|
Thanks, @roryclaydon1994, again for the careful and thoughtful review! This iteration helped me identify a few additional improvements beyond the specific comments, including adding an explicit type check for |
roryclaydon1994
deleted the
241-architecture-extend-quality-control-configuration
branch
August 18, 2026 14:06
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Extends the Quality Control configuration framework to support configurable execution resources and metric resource requirements.
The configuration now allows quality metrics to declare the resources they require, while the configuration loader validates that those resources are available and that enabled rejection policies reference enabled quality metrics.
This provides the configuration foundation required by the Quality Control pipeline orchestration.
closes #241
What’s changed
ResourcesConfig.required_resourcestoQualityMetricConfigso metrics can declare the resources required for their computation.resourcessectioninference_configfrom metric-specificparamsto theresourcessection.paramsarguments and values, while preservingparamsas the mechanism for metric-specific configuration.QualityControlConfigHandler.load().How to test / verify
pytest tests/quality_control/test_config.pyValueError.Scope
This PR is part of the Quality Control architecture introduced under #226 and provides configuration plumbing required by #240 (Quality Control pipeline orchestration).
Changelog
Reviewer Checklist
develop, ormainfor release PRs)ruff)Next Steps / Notes
The resource configuration is intentionally generic. The current implementation supports declaring resources such as standard PSF inference configurations, while the pipeline orchestration will resolve and prepare the resources required by enabled metrics.
Resource generation, execution-context construction, and metric computation remain part of the Quality Control pipeline implementation in #240 and subsequent metric-specific work.